{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Conditional Generative Adversarial Networks\n",
    "\n",
    "In [deep convolutional generative adversarial networks (DCGAN)](./dcgan.ipynb), \n",
    "we introduced how to generate human face images from random noise. With a DCGAN, we can generate images from random vectors, but what kind of images do we achieve? Can we specify it is a woman or man face?\n",
    "\n",
    "In this notebook, we introduce [conditional GAN](https://arxiv.org/abs/1411.1784), which accepts label as part of the input for both generator and discriminator. With conditional GAN, you can choose which kind of data to generate with corresponding label. We'll train on MNIST dataset."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from __future__ import print_function\n",
    "import numpy as np\n",
    "from matplotlib import pyplot as plt\n",
    "\n",
    "import mxnet as mx\n",
    "from mxnet import gluon, test_utils, autograd\n",
    "from mxnet import ndarray as nd\n",
    "from mxnet.gluon import nn, utils"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Set training parameters"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "epochs = 1000\n",
    "batch_size = 64\n",
    "label_size = 10\n",
    "latent_z_size = 100\n",
    "hidden_units = 128\n",
    "img_wd = 28\n",
    "img_ht = 28\n",
    "\n",
    "use_gpu = True\n",
    "ctx = mx.gpu() if use_gpu else mx.cpu()\n",
    "\n",
    "lr = 0.001"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Download and preprocess the MNIST Dataset"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# Pixel values of mnist image are normalized to be from 0 to 1.\n",
    "mnist_data = test_utils.get_mnist()\n",
    "train_data = mnist_data['train_data']\n",
    "train_label = nd.one_hot(nd.array(mnist_data['train_label']), 10)\n",
    "train_iter = mx.io.NDArrayIter(data=train_data, label=train_label, batch_size=batch_size)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Visualize 4 images:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXQAAABrCAYAAABnlHmpAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAADAFJREFUeJzt3XnoVNUbx/HPN6MybNNIg/Zsk6y0\nVWhVQbI0WiiljBZs+cMW24xWotKiVEwiqZBo3wzLyqzMogKxUjQ0tcWijWi3tGyxf37PM8/8Zhzn\nO9/vnZl75v36x4czc2fud7g+nHvOuc9pW79+vQAA+bdJo08AANA5SOgAkAgSOgAkgoQOAIkgoQNA\nIkjoAJAIEjoAJIKEDgCJIKEDQCI2reeXtbW18VjqRqxfv76tluP4bTeu1t9W4vetBtdudqr9bemh\nA0AiSOgAkAgSOgAkgoQOAImo66QoAOTV3nvvLUmaPXu2t3Xp0sXjXXfdte7n9P/ooQNAIkjoAJAI\nhlwAYAPuuecej8844wxJUvfu3b1t1qxZdT+nSuihA0Ai2uq5p2gjngjr06ePJOnEE0/0tgsuuECS\ntGDBAm9buHBhybGTJ0/2eN26dVmdYhGetssOT4pmK+/Xbs+ePSVJM2bM8LYjjjjCY8uVH374obcN\nGjTI4x9++CGzc+NJUQBoMSR0AEhEkkMuF154ocd33XWXJKlbt27t/pyBAwd6/MYbb3T8xKqQ99vW\nZpblkItdXzZxJkl//PGHxwcffLAkaauttvK2M888U5I0b948b/vqq6+qOp9vv/3W45kzZ3r83nvv\nVXV8FvJ47dracqmQK4YOHeptbW2FP2ncuHGSin/jZssL9NABIBEkdABIRJJDLnGd6LJlyyRJO+yw\nQ7s/5+eff/Y43krPmTOnA2dXWR5vW/MiyyGXO++8U5J05ZVX1voVNfv33389Xrp0qSTp8ccf97YY\nr1q1KrPzyOO1G1exvP322yWvxyGXs846S1Lx71kvDLkAQItJsoceXXTRRZKku+++29u23HJLSdIX\nX3zhbbvsskvFz5k0aZLHY8eO7cxTLJLHXk4WYqGjrl27SpJGjhzpbRdffHHJMS+++KLH5557bsnr\nWfbQP/74Y0nSHnvsUfFz4lrlxYsXV/Xdy5cv93ifffaRJG277bbe1q9fv4rHDxs2zOP4G3W2PF27\n5QptlSuudcopp3gcJ5/rjR46ALQYEjoAJCL54lz33XefpMLQiyQdeOCBkqRff/216s+ZOnVq554Y\n3ODBgz22W9w4vLLNNttIKjx6vSFxgqvehgwZIql4XfOKFStK3rdmzRqPv/nmm5q/L65nX7Jkicfl\nhg6HDx/ucZZDLnkyatQoScW/10svvSSpOFdU+1xAs6CHDgCJSL6Hbm699VaPr7vuOknSQQcdVPXx\nm222WaefUyt64IEHJEl9+/b1tkMPPbTiMatXr5YkPfroo94WC6vZMrL4ZGa9ffLJJ0X/Zi0WmyvX\nK//zzz89vv/+++tyTs3u3Xff9dj+78dlnJdffrmk/PXKI3roAJAIEjoAJKJlhlyeeeYZj+2JsPjE\nZxwCKCcO2Zx22mmdfHbp6dGjh8fjx4/3+LzzzpMk/fjjj972/vvvezxhwgRJxTWn165dK6n4uYFW\nEof7pkyZIkk6++yzKx4zYMAAjxctWpTNieXASSed5PHhhx/usU2wP/30097WyCG7zkIPHQASQUIH\ngES0zJCL1Z6WCuvQ999//6qPL1e4Bxt2ww03eHz++ed7bJvu2kojSfrtt9/qd2I5ctxxx0kqrJmW\npHPOOafkfX/99ZfHl1xyiSTpo48+yvbkmpyVRjjqqKMqvu+nn37y+Msvv6zqsy+99FKPd95555LX\nG1GgzdBDB4BEJNlD33fffT1+7rnnJEm9e/f2tk03bf+f/fzzz3f8xBJjRc6uueYab7Pe5GWXXeZt\ncVeXV155RVIaE1BZOOywwzy2SfsuXbpUPCY+QWsTx//8808GZ5cf9vfbTlGStMkmhf6rlRx+6623\nKn6OrU2PxowZ43G5gl5XXHGFxzvttJOk+q1tp4cOAIkgoQNAIpIcctlvv/083n333SXVNswSxVuv\neMvVyq6//npJxUMuTz31lKTiNf4Mr1Tv9NNP93hjQy0mrlO34ltxI+MXXnjBYxuCjOv8U3TMMcdI\nKp4UjTs72dDU999/X3JsLAkSj49Fzszvv//usU2qWs16qfD8y4gRI7zt888/r/KvaD966ACQiCR7\n6NYLkaSrr75aknTHHXd42xZbbNHuz9xxxx07fmKJufbaayUVT8o1Q6GsPJsxY4bHdqcZi5dtv/32\nVX3OIYccUja+6aabJEmTJ0/2NtsP9bvvvqvhjJtHLClsd+bR119/7fHDDz8sqbDTlFQofXzVVVd5\nW3zS1Hrz8e4z7oRmZZ7nzp1b0lYv9NABIBEkdABIRPKbRJvjjz/e47jBromTprY70dZbb+1tzz77\nrMdZFufK00a78+fPl1R8S2/rbePToa+++mp9T2wDstwkOkux3rkNufTs2dPb4kbGVvysra36P/XN\nN9+UJA0aNMjb4gRitRp97cb/43Ei2Nxyyy0lcfwdrW780KFDvS0+xWzDNPFJ0L322stjK/QVh2ft\nmI4upGCTaABoMSR0AEhEywy5bEy8Rb355pslSTfeeKO3xa3F7NY0i/Wkjb5tjax+9MKFC71t3bp1\nHnfv3l1SoSCUVCjKFW9VYx3qRhaNyuuQS3tYEbp4ix/LCVQybtw4j23lS3s0+tqNz0PcdtttJa+X\nexblnXfe8ThepyYOQ9nQVNyMvFzRvriCqLMKdTHkAgAtJsl16LWIT9vFnrmJJUpTLHxkEzmzZs3y\nNpuMi0/JPvLIIx7brkM2iSwVeujdunXzNuvJI3u2kfaTTz7pba+99prHRx999AaPjQXs8igudrA7\n7pkzZ5Z9rz0Nuttuu5UcE4trWa9cKqxTf+yxx0qOicfFHnq90UMHgESQ0AEgEQy5/E/cBLqcBx98\n0ONqdzbJkw8++EBS8dp7m2SKwyzlxB1cTLzNT70QVDP6+++/PY6bcFcaclmxYkWm51RPtthjY4s+\n4np7e+8BBxzgbXFjcisZ8tlnn3lbLN71yy+/dOCMOwc9dABIBAkdABKRq3XoPXr0kCRNnz7d26y6\nn/3bHvER3bg+Og47mD333NPjTz/9tN3fVa1GreW1yolW41ySunbtWvGYlStXSip+/NnW5p966qne\nZsM5jdZs69Dt+hs9erS3xevQasvXItZSt23/JGngwIEl77XhmfhaLZuiN3od+sbWhx955JEe2yqX\nCRMmeFtcmRXOzWOrthg36n755ZdrP+F2YB06ALSYXE2KTpkyRZI0bNgwb7O1obHWcdyQ1eodx81i\n7RirlS6V75XHWsfx81M0fvx4ScXr7fv16ydJGjx4cNljtttuO0mFXXKkwpNxsc40Cnr16uXx7Nmz\nJUl9+/b1NvtNa2XFpsaOHett5Xrl0bJlyyTV1itvJvHaXbNmjaTCRuZS8VOh1Y5MrF692mO7Y6pX\nr7wW9NABIBEkdABIRK4mRW3SY+LEid42YMCAkvetWrXK46VLl0oqXi8at6oy8Xewiam49VfcDDZL\njZ5YSlkzTIo+8cQTHscNoU3//v09Xr58uSRp7dq1Je+LE9Zx6NCGWspd41Jhki8OJdgQZnzMvRbN\ndO2ecMIJkoqHno499liPy+W9hx56SJK0ZMkSb4uF6Tr6+3QEk6IA0GJy1UM3cbLSJt/uvffeDn2m\nFZqSCssjG6GZejmpaYYeelyiOG3atIrvtd5huScQ4+bDNnldDStrfPLJJ3vb66+/XvXxlXDtZoce\nOgC0GBI6ACQil0Mu0eabby6peNecyG5HR44cWfJavJWNa3Ub+WQjt63ZaYYhl1h/+/bbb5ckjRgx\nojM+ukgszhXrc9tm57bBd2fi2s0OQy4A0GJI6ACQiNwPuaSG29bsNMOQS2TDhXHFSRz6s/rkw4cP\nLzl2Q5ttz507t+T1RYsWdfxkq8C1mx2GXACgxdBDbzL0crLTbD301HDtZoceOgC0GBI6ACSChA4A\niSChA0AiSOgAkAgSOgAkgoQOAImo6zp0AEB26KEDQCJI6ACQCBI6ACSChA4AiSChA0AiSOgAkAgS\nOgAkgoQOAIkgoQNAIkjoAJAIEjoAJIKEDgCJIKEDQCJI6ACQCBI6ACSChA4AiSChA0AiSOgAkAgS\nOgAkgoQOAIkgoQNAIkjoAJAIEjoAJOI/kwYvsXncTkUAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f428ca64e90>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "def visualize(img_arr):\n",
    "    plt.imshow((img_arr.asnumpy().reshape(img_wd, img_ht) * 255).astype(np.uint8), cmap='gray')\n",
    "    plt.axis('off')\n",
    "\n",
    "for i in range(4):\n",
    "    plt.subplot(1,4,i+1)\n",
    "    visualize(nd.array(train_data[i + 10]))\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Defining the networks\n",
    "\n",
    "Input of generator is the concatenation of random noise vector and digit label one hot vector. It is followed by a fully connected layer and relu activation function. Output layer is composed by another fully connected layer and sigmoid activation function.\n",
    "\n",
    "Similar to generator, input of discriminator is the concatenation of flatten image vector and digit label. It is followed by a fully connected layer and relu activation function. Output layer is composed by another fully connected layer and sigmoid activation function. In this tutorial we don't apply sigmoid function to output to make training process more numerically stable.\n",
    "\n",
    "![](../img/cgan.png \"Conditional GAN Architecture\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "w_init = mx.init.Xavier()\n",
    "\n",
    "# Build the generator\n",
    "netG = nn.HybridSequential()\n",
    "with netG.name_scope():\n",
    "    netG.add(nn.Dense(units=hidden_units, activation='relu', weight_initializer=w_init))\n",
    "    netG.add(nn.Dense(units=img_wd * img_ht, activation='sigmoid', weight_initializer=w_init))\n",
    "\n",
    "# Build the discriminator\n",
    "netD = nn.HybridSequential()\n",
    "with netD.name_scope():\n",
    "    netD.add(nn.Dense(units=hidden_units, activation='relu', weight_initializer=w_init))\n",
    "    netD.add(nn.Dense(units=1, weight_initializer=w_init))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Setup Loss Function and Optimizer\n",
    "We use binary cross-entropy as our loss function and use the Adam optimizer. We initialize the network's parameters by sampling from a normal distribution."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# Loss\n",
    "loss = gluon.loss.SigmoidBinaryCrossEntropyLoss()\n",
    "\n",
    "# Initialize the generator and the discriminator\n",
    "netG.initialize(ctx=ctx)\n",
    "netD.initialize(ctx=ctx)\n",
    "\n",
    "# Trainer for the generator and the discriminator\n",
    "trainerG = gluon.Trainer(netG.collect_params(), 'adam', {'learning_rate': lr})\n",
    "trainerD = gluon.Trainer(netD.collect_params(), 'adam', {'learning_rate': lr})"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Training Loop\n",
    "We recommend that you use a GPU for training this model. After a few epochs, we can see human-face-like images are generated."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from datetime import datetime\n",
    "import time\n",
    "import logging\n",
    "\n",
    "real_label = nd.ones((batch_size,), ctx=ctx)\n",
    "fake_label = nd.zeros((batch_size,),ctx=ctx)\n",
    "\n",
    "def facc(label, pred):\n",
    "    pred = pred.ravel()\n",
    "    label = label.ravel()\n",
    "    return ((pred > 0.5) == label).mean()\n",
    "metric = mx.metric.CustomMetric(facc)\n",
    "\n",
    "stamp =  datetime.now().strftime('%Y_%m_%d-%H_%M')\n",
    "logging.basicConfig(level=logging.INFO)\n",
    "\n",
    "for epoch in range(epochs):\n",
    "    tic = time.time()\n",
    "    btic = time.time()\n",
    "    train_iter.reset()\n",
    "    iter = 0\n",
    "    for batch in train_iter:\n",
    "        ############################\n",
    "        # (1) Update D network: maximize log(D(x)) + log(1 - D(G(z)))\n",
    "        ###########################\n",
    "        data = batch.data[0].as_in_context(ctx)\n",
    "        label = batch.label[0].as_in_context(ctx)\n",
    "        latent_z = mx.nd.random_normal(0, 1, shape=(batch_size, latent_z_size), ctx=ctx)\n",
    "        D_input = nd.concat(data.reshape((data.shape[0], -1)), label)\n",
    "        G_input = nd.concat(latent_z, label)\n",
    "\n",
    "        with autograd.record():\n",
    "            # train with real image\n",
    "            output = netD(D_input)\n",
    "            errD_real = loss(output, real_label)\n",
    "            metric.update([real_label,], [output,])\n",
    "\n",
    "            # train with fake image\n",
    "            fake = netG(G_input)\n",
    "            D_fake_input = nd.concat(fake.reshape((fake.shape[0], -1)), label)\n",
    "            output = netD(D_fake_input.detach())\n",
    "            errD_fake = loss(output, fake_label)\n",
    "            errD = errD_real + errD_fake\n",
    "            errD.backward()\n",
    "            metric.update([fake_label,], [output,])\n",
    "\n",
    "        trainerD.step(batch.data[0].shape[0])\n",
    "\n",
    "        ############################\n",
    "        # (2) Update G network: maximize log(D(G(z)))\n",
    "        ###########################\n",
    "        with autograd.record():\n",
    "            fake = netG(G_input)\n",
    "            D_fake_input = nd.concat(fake.reshape((fake.shape[0], -1)), label)\n",
    "            output = netD(D_fake_input)\n",
    "            errG = loss(output, real_label)\n",
    "            errG.backward()\n",
    "\n",
    "        trainerG.step(batch.data[0].shape[0])\n",
    "\n",
    "        # Print log infomation every ten batches\n",
    "        if iter % 10 == 0:\n",
    "            name, acc = metric.get()\n",
    "            logging.info('speed: {} samples/s'.format(batch_size / (time.time() - btic)))\n",
    "            logging.info('discriminator loss = %f, generator loss = %f, binary training acc = %f at iter %d epoch %d' \n",
    "                     %(nd.mean(errD).asscalar(), \n",
    "                       nd.mean(errG).asscalar(), acc, iter, epoch))\n",
    "        iter = iter + 1\n",
    "        btic = time.time()\n",
    "\n",
    "    name, acc = metric.get()\n",
    "    metric.reset()\n",
    "    logging.info('\\nbinary training acc at epoch %d: %s=%f' % (epoch, name, acc))\n",
    "    logging.info('time: %f' % (time.time() - tic))\n",
    "\n",
    "    # Visualize one generated image for each epoch\n",
    "    fake_img = fake[0]\n",
    "    visualize(fake_img)\n",
    "    plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Results\n",
    "Given a trained generator, we can generate some images of digits."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAT8AAAD8CAYAAAABraMFAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAIABJREFUeJztnXl0VOX5x7937qzJTHZCgAARU0kh\nQpDIJmVRAa0VQS1tKipowVKFSq2itaKVoiIiCrYo4A+Fg0oBQSkCQmVTZCdsgbAEkpA9k0wyzHpn\n7vP7gybHmARJZsKd5fmc855jZsY7z/3wznPfu7zvIxARGIZhwg2V0gEwDMMoASc/hmHCEk5+DMOE\nJZz8GIYJSzj5MQwTlnDyYxgmLOHkxzBMWMLJj2GYsISTH8MwYYn6en6ZIAh+m05CRIK/thUKsNu2\nhf22HUq55ZEfwzBhCSc/hmHCkoBLfgkJCfj000+xfPlyLFu2DLIsY9OmTUqHFdQkJCRgxYoVkGUZ\nsizD6/UqHVJIMXr0aDz88MM4fPgw3G43iAgmkwlRUVFKhxYyjB07Fnl5eTh79ixmzZqFqqoq6PV6\n3zZKRNetAaCrNbVaTS6Xi4iI1q5dS3379iVJksjlclGHDh0afPZ6xh0M7WperVYr/ZiCggIyGAxN\nfl7pfQnE1pzbZ599lmRZbuSXiMjhcJBarWa/rXRb1/r160cWi4W8Xi/t2rWLlixZQh6PhzZs2OCT\nW8V38n8XO+n3v/89SZJERESyLNOoUaMoJiaGXnnlFSovL6eXXnqJO1AL3er1evr000/JYrGQ1Wql\nmpoa8nq9VEdNTQ117tyZf5yt9Pvss8/Snj17qLCwkEpLSyk/P5/mzZtH77//PsmyTLW1tbR7925q\n164d+22h27q2evVqkiSJzGYzrVixgkwmE6lUKpoyZQrl5+f75DYgTntVKhVefvlleL1eHDlyBCtX\nroRarYYkSViwYAFsNhsGDhyodJhBh1arRfv27WEwGGA2m3HPPfegqqoKNTU1AICoqCj87W9/UzjK\n4KZjx47wer1YsmQJfv7zn2PRokXIyckBAOj1enTv3h21tbUKRxm89O3bF263G2vWrME777yDiIgI\nEBHWrFmD3NxcvPjii63fuNIZ/pNPPqHy8nIiIvrqq69IFMVGp8IXL16kc+fO8dGzhW4PHz5MhYWF\ntHLlygYjPFEUaenSpSTLMsmyTMuWLWO3rfD79ttvU0lJCc2YMaP+EoJKpaLevXuTRqOh3NxcKigo\noNjYWPbbQrcAaO7cueT1emnixImN3ktISKDjx4/TZ5991mq31/U5vx8jCAJ69+6NwsJCfPTRRzh8\n+HCji/F6vR6JiYkoLi5WKMrgJT09HbIsY9asWSgqKqp/XZZlvPzyyxgzZgzi4uLQuXNnBaMMXkaP\nHo2oqCgsWLAAbrcbwBW3R48eBQBYrVZs3LgRoigqGWZQkpaWhvT0dDz44IPYvn17o/cjIyMRGRmJ\ns2fPtv5LlMrwgiDQnDlzyOVyUa9evZo951epVERE9N133/HR8xrdGo1G+u9//0tERGfOnGnSq1ar\npcLCQvJ4PDR48GB22wK/Op2O7r33XiIiKisra9KvWq2myspKOnz4ME2aNIn9XqNbAGQwGKioqIjW\nrVvXbF7o1asXud1uGjt2bPBd84uLi8Ojjz4KlUqFixcvNvu59PR0SJKE3Nzc6xdckHPvvffC6XTC\nYrHg9ttvb/IzkiTh2LFjkGUZBw4cuM4RBjd6vR49e/YEEWHbtm1Nfkar1eLEiRNIT0/H5cuXr3OE\nwU1ERATi4+NhtVqbfF+j0eC9995DTk6OT4/BKZb84uPjERcXh5qammZ3UqVSYdCgQZBlGbfddtt1\njjB4OXLkCEwmE8xmM6qrq5v8jFqtRnJyMpxOJzwez3WOMLiRJAk9e/aEJElISEho8jN2ux0PPfQQ\nXC4XJk+efJ0jDG5+//vfg4hw+PDhJt8fNGgQUlJSsGnTJp+eWVXsml9aWhq+/fZbPP/883VD30bk\n5+fDYrFg9uzZmDdv3nWOMHjp0KEDampqYDabccMNN+DEiRONPlN3jernP/85P/TcQrxeL9577z30\n7dsXERERiIyMhM1ma/CZ6OhoPPbYY9BqtQpFGZzo9Xp4PB5s3LgRCxYsaPIzO3bsgN1ux/z58yFJ\nUqu/S7GR37Rp0+ByuRAREdHk+6mpqTh37hzOnTuH+fPnw+FwXOcIg5f9+/dj2LBhuO2223Drrbei\nY8eO0Ol0AK7cZDIajdizZw+8Xi8KCgoUjjb46Nq1K7p164bY2FicPXsWLpcLgtBwPv3ly5cxZswY\nOBwO5OXlKRRp8CFJErxeL2677TakpaXVv67VajF06FDk5eXB7XZj7ty5qKio8Om7hOZGXW3BD1dv\nKCwsRFFREY4fP44//OEPkGUZAwcOxPbt26HVakFEMBqNsNvtTW6LeGWMBvx4ZYx+/frhtddew+DB\ng+sTX92FXqfTidraWtxyyy0oKSlptC1225gf+42Pj8fzzz+Pvn374rvvvoNOpwMRYcqUKdBoNPV9\n2GAwNBqdsN+G/NhtREQEqqqqIMsybr/9dqxevRrJyckAgJqaGsTExDS7rZa4Vey0NyEhAXFxcejX\nrx/uv/9+REREQK/Xg4ggyzJ27drVbOJjfprDhw/j1KlT6Nu3b33yczgcyMnJQadOndCpU6dmLzcw\nP01VVRVUKhV+9rOfYeDAgbh8+TJkWYZarYZOp4PT6cSiRYt8Oi0LV+x2OxwOB44dO4bPPvsMnTp1\ngizLKCgoaDAa9BXFRn6+wkfPhrDbtqUpv+3bt8f777+PwYMHQ6vVwu1246uvvsL8+fORnZ3d7LbY\nb0OU6ruc/EIEdtu2sN+2gxczZRiGuY5c15EfwzBMoMAjP4ZhwhJOfgzDhCVcvS1EYLdtC/ttO/iG\nB8MwzHWEkx/DMGFJwCU/k8mEnJwc35anZprk1KlTOHbsGEwmk9KhhCQajQYvvPACXC4XNBqN0uGE\nHCkpKbBarWjfvr1fthdwya9bt24gIq570Abk5uZi0aJFPG2wjYiJicFzzz0HtVqNVatWKR1OyPH4\n448jMjIS//rXv/yyvYBLfhMnTkS3bt1w6NAhpUMJOW6++WbEx8crHUbIkpOTg+joaAiCUL9QL+Mf\nYmJiMGnSJHg8HowbN84v2wy4fx1RFKFWq1FaWqp0KCGHXq/HwYMHlQ4jZKkrCk9EcDgckGVZ6ZBC\nhrFjxyImJgYul8tv608GXPKLj4+HWq2G2WxWOpSQo6amBk8++SQvXtoGCIKA6OhoqFQqSJLEic/P\nzJkzBzqdzq8Hb0WrtzXF8OHDAVwZ5tbVl2X8Q0JCQv0Kzox/UalUUKuv/Jx41Od/oqOjIcsyPv74\nY79tM+CSX0JCAsrLyxuUWmT8Q1xcHNfraCP0en19icqXXnpJ4WhCi969e0Or1cJqteLf//6337Yb\ncKe9arUa0dHR/CNtA0RRvOoquEzrEEURAwYMAAC4XC785z//UTii0MFoNGLRokVwOp2YPHmyX59U\nCLiR34kTJ/iUoY24dOkSvv/+e6XDCDkiIyPx6KOPwuv14sSJE8jPz1c6pJDB4/Hg5MmTSEtLa7ZM\naGsJuJHfz3/+c+48bURiYiK++OILpcMIObp06YI777wTwJUCPLxMnP9ITExEv379MHjwYL/fBA24\n5OdwODBkyBClwwhJNBoNnnrqKaXDCDl+/etfIzIyEiqVCkeOHFE6nJBiyZIl6NWrF0pKSvx+UAm4\n5FdbW4tly5YpHUZIUlVVhcLCQn741s988MEHMJvNcDqdsNlsPLXNTwiCgJEjRwIAqqur/b79gPsV\nXL58Ge+//77SYYQkgiAgNze3UY1Zxjf0ej0qKipgs9kQGRmpdDghQ91Buq1qdnMBoxCB3bYtV/Or\n0Wjw/PPPY9y4cWjXrh06dOhw1VM09tsQrt7WQrgDNYTdti3st+3gxUwZhmGuI1y9jWGYsIRHfgzD\nhCWc/BiGCUs4+TEME5Zw6coQgd22Ley37eC7vf8jJSUF9957L1JSUlBcXIy4uDilQ2KYqyIIAqZN\nm4ZbbrkF/fr1w5gxY5CYmMgPkwc4AZX8BEFAVFRU/Yqtly9fxqZNm5QOK+i5ePEiPB4PZFlGXl4e\niAhEhB07digdWtAjiiJ27tyJPn36YM6cOdi+fTvWrl2LtWvX8gIHfkKj0eCtt96C1WoFEcHj8SA9\nPd3n7QZU8hs4cCDefvtt9O/fHxs3bkSnTp3Qo0cPaLVapUMLWrRaLSIiInDy5En06tULjz32GGpq\nalBRUcGjah8RBAF33nknkpKScMstt+Czzz7D+fPnUVxcjIqKChgMBqVDDAkWLVqEP//5zzAajQCu\nTHv729/+5vuG60YB16MBoObawoULKTs7m15//XUaOXIkaTQakiSJjh8/TiaTqdHnr2fcwdCa85qS\nkkJvvfUWdezYscHrNpuN3G53k/+P0vsSiK0pTwkJCbR+/Xp64IEHKCIigv537YpiY2PJbrfT5MmT\n2W8r3QIgQRDo5MmTlJeXR+3btydBEEgQBCIislqtPrsNiJ0EQBqNhubNm0fR0dEEgLRaLdlsNior\nK6vvVNyBWuf2xhtvJKPR2KBTbd26lUpKSvjH6YPf0aNHU3x8PImi2KCP6vV6stlstGLFCvbbSrcA\nqH///vTpp5/S4sWL6e677yaVSkUqlYq8Xi+dOnUqdJLfj1tiYiI5HA56/fXXuQP52a0oiuTxeOjC\nhQvs1ge/KpWKRFFs9NrUqVPJ4/HQtm3b2G8r3TY3EvzNb35DTqeThgwZ4rPbgFvGHriyRNDkyZMh\nCAL279+vdDghh1arxeXLlzFx4kSlQwlqmiq3oNVqcfjwYTgcDr8V12au3Fjq3r07tFotLl26hIsX\nL/q8zYBJfoIgoGvXrkhISMB///tfREREYMyYMfjqq6+UDi3o0Wg0MBgMiIyMxOzZs3H77bcjKSkJ\nLpdL6dBCgtWrV+PWW2/F9OnT0b9/f/Ts2ROyLKN3797Yvn270uGFBJGRkTh58iQA+O0RIsWTX/v2\n7XH33XfD6XTihhtuwPjx4xEREQGLxYKNGzcqHV5QExkZCYPBgOrqaoiiiG+//RbJycmwWq3weDx1\npxxMKxk3bhymTp2K2267DQDw0UcfoaKiAsnJybDZbOjatSsiIiIwatQorFu3TuFog5u6g4hfqzoq\nfW6fkZFBf//738npdFJtbS3Jskxer5emTZt21fN/pa9TBFprytGQIUNIkiRat24dlZSUEBGR1+sl\nj8dDGo2G3bbQryRJDTytWrWKPB4PSZJElZWVtGPHDrLb7STLMlmtVnr00Uepf//+dPjwYfb7E26v\n1jp27EhERLIs06RJk/yWFxTdSYPBQMXFxXTmzBnKy8sjm81GkiSR2+2mDz74gEaNGtXgLiV3oJZ1\nIKfTSQUFBdSuXTvq3r07paen09tvv01ERMuXL6fu3btTampqo7vpSu9LILam/Nrtdjp37hxptVoS\nBIFqampIlmUiIqqqqqJDhw7Rm2++SRkZGey3hW5FUaQXX3yR9u7dS1arlR5++OH69zQaDS1dujR4\nk58gCJSZmUlOp5N27dpFTqez/oiZm5tLX3/9NZWVldHKlSs5+bWyAxUWFtKKFStIpVLRxIkT6b//\n/S9lZ2eTLMskSRJ99NFHlJWVRQaDgd22wm/d4yx6vZ4+//xzkmWZXC4XWSwWOnnyJM2dO5fat29P\nnTt3Zr8tdLt7924iIvrkk0+orKyMzGYzJSQk0NChQ+nixYtkt9t9fgROsZ0UBIHuu+8+OnnyJHm9\nXkpLSyO9Xt9gVHj8+HHq2rUrJ79WdqALFy5QcXEx5ebmkiRJNG3aNBo7dixlZWXRSy+9RFOmTCGt\nVstuW+m3vLycvF4veb1ekiSpydGIKIoUExPDflvgNikpqd5rWVkZuVwu6t69O7322mvk8Xjo6aef\npsTExOAd+QFXnuXLysqijz/+uNHzUpGRkfTSSy81ObuDO9BPuwVAGzZsoJKSEpIkiWRZJpPJ1OTR\nkt22zm9mZiZZLBaqra0ll8vVrE+1Ws1+W+A2KyuLXC4XuVyu+rOUf/7zn1RcXOzX69VcwChEYLdt\nC/ttO3hJK4ZhmOsIJz+GYcISrt7GMExYwiM/hmHCEk5+DMOEJZz8GIYJS7h6W4jAbtsW9tt28KMu\nDMMw1xHFk59Go0F6ejqWL1+O2tpaeDwefPnll0qHFRIMHjwYixYtwuHDh2E2m+HxeJCXl4dly5Yp\nHVpIoFKp0K5dO8yfPx+5ubnwer1YsmQJ3n//faVDC3p0Oh1mz54Nh8MBSZLgcrmwd+9euN1uiKLo\nny9ReopQXl4eeb1e+jFVVVU8BcsHt6Io0o4dO0iSpAZ+PR4PWa1WioiIYLc++FWpVDRmzBiy2Wz1\nK7nUIcsyrVy5ssl50+z3p90CV+b2L1iwgKxWK9lsNurWrRtlZWWRxWKhVatWNTtNs0Xfq+ROmkym\n+g4zadIkeuCBBxp0orS0tEZzfrkDXZtblUpF48aNo4yMDLrxxhspNTWVRFEkrVZLS5cuJSKilJQU\ndttKv7fccgsRXVkfcfv27XTTTTdRRkYG/eEPf6CTJ0+SLMvk8XjYbyvc/rClp6dTly5d6uullJaW\n0vLlyyktLS24k59Wq6U6NBoNqdVqcrvdRHTl6MkjP/90oOZ+uP7oQOHSfuyoa9euVFhYSEVFRfUl\nFQVBIJPJRC+88ALVwX5b7vaHTa1Wk16vJ1EU6Z577iG3293siLqlbhW95idJEi5cuIB33nkHkiTB\n4/FAFEXY7XbU1NTg4YcfVjK8kEOtVmPWrFnYt28fKisrcfr0aaVDClouXbqE1atX45577qn/ManV\najz11FOYNm0avF4vLl++rHSYQU98fDxmzpyJgwcPon///vjPf/4DSZL8sm1Fa3gQEfLy8tCxY0fo\ndDpMmjQJbrcbOp0Obrcbffr0wb59+3DmzBklwwxaBEEAESE2NhZdu3bFpk2bkJiYCJVKhZkzZyod\nXlDj9Xqxdu1aAFcOKunp6Rg6dChmzJiB6OhonDhxAlqtVuEogxe1Wg1ZlrFp0yYYjUZ07doVqamp\nqK6urhst+v4dftmKD7zxxhv4+uuvMWjQIGi1WsiyjJycHLhcLixduhTbtm1D165d/bbD4cTIkSOx\nadMmAFcS4T/+8Q9s2bIF6enpsFgsEEURXq9X4SiDl169euGf//wngCsH8pqaGqhUKlRUVECtVkOr\n1SIqKgqxsbHIz89XONrgQhRF/OIXv0BqaioEQYDb7YZWq4XdbodGo/HL6E/xR1127NiB7OxszJ07\nF2+//TaAK1n/q6++wrFjxxAbG4v+/fvXf16lUjzkoOGOO+6AzWaD1WrF0aNH8eabb6KyshKCIODv\nf/87UlNTlQ4xqOnevTsuX76MwsJC7NmzB9u3b8fOnTsxbNgw3HDDDYiLi4PRaITZbFY61KBDEARU\nVFTgmWeewdixY5Gbmwu3243Y2Fjcf//9iIiI8P1LAuHCplqtJlEUSaPRkFarpaioKBoxYgSZzWaq\nqqqqLwDzw9vbSl+kDbTWlFe9Xk8Gg4Gio6MpMjKy/i7w4MGDyeFw0KBBg/iCvA9+6+5ARkREUHx8\nfP1ND1EU6ezZs2S1WulPf/oTtWvXjv220G1dXkhMTCSj0Uivv/46ff/992S1WunDDz+kP/3pT8F9\nt/eHLSkpqdGzO7W1tWS32+ndd99tdGdS6X+wQGtXc/vj1qlTJ6qurqbJkydz8vPRr1arbfLuY13y\nW7lyZaM6NErvS6C1a+23AwYMoLNnz5LZbKZTp04F993eOiIjI5Gbm4tdu3Y1eN1sNtcX3eabHv5B\nFEW8+eabEATBb3fNwhWdTodu3bohMjKywetGoxHt27eHwWDAJ598AkHgqbwtRaVSNfCWnp6O1NRU\nOJ1OuN1uHDhwwPcvCYQMHxERQXWUlpZSUVER1dTUkMfjoYKCAoqLi+PRSSvcRkREUK9eveqfoYyI\niKDXXnuNamtraffu3XT33XfzyM8HvzNnzqTCwkIaNWoUpaSk0MaNG6miooIKCwvJYrHQq6++Wn8q\nzH6vzW3d5a+qqirKz8+nrKwsWrFiBZ08eZK++eYbOnbsGP3zn/+k5OTk0DntraOuZB0RUW1tLS1Y\nsIB/oK10O3PmTDp16hQdPHiQ3nnnHbLb7STLMjkcDpo3bx499NBD7NYHv2fOnKmvjGc2m0mWZZJl\nmUpLS2nWrFmNSlay32tzKwgCWa3W+jrIdVM0a2trad26dc1WdGyp24Cp3qZWq9GvXz9ER0ejoKAA\nTz/9NLZt24YtW7bAYrE0+jzxskANaMrtjTfeiDlz5mDgwIEwm81ISkrCoUOH8Pe//x179+5tdlvs\ntjHN9d0xY8bgz3/+M4xGI/bt24evv/4a33zzDWpqaprdFvttiFJLWgVM8msp3IEawm7bFvbbdvB6\nfgzDMNcRrt7GMExYwiM/hmHCEk5+DMOEJZz8GIYJS7h6W4jAbtsW9tt28N1ehmGY6wgnvzBFp9Nh\n3759SElJ4bmnbYBOp8O2bduQlJSkdCghhyAIuOmmm7Bt2zafKrkFTPKLiYlB+/btG7x24cIFXmyz\njXj33Xdx4403olOnTrxGog8IggCj0djoAPLRRx+hT58+KC8vVyiy0EEURcTFxdX/bTAYsH//fiQm\nJkKn07V6uwHR6zUaDY4dOwaXy9Xg9aSkJGzYsEGhqEIDlUqF5557DgsXLkS7du3qX3/00UcBAHv2\n7OEDTCuJiorCM888gxEjRuDHz8uWlpbi0KFDkGVZoehChxMnTuCRRx6p/7uwsBB79+5Fr169YLfb\nW71dxZOfSqVCXFwczGYzHA5H/es6nQ5Hjx7F1KlTFYwu+NHr9XjhhRdw7733oqKiosF7+/bt4zoT\nPhAREYG0tDRs3ry50XuZmZn4xz/+oUBUocXs2bORnJyMTz/9tP41jUbjn+JQSq+MkZSURH/5y19I\no9HUv6bRaCg3N5fi4+P9snpDOLSmHHXv3p1effVVKi8vpzlz5hBwZfXhKVOmUF5eHiUlJbFbH/x+\n9913ZLPZGr0eFxdHW7ZsadCn2W/L3AJXViJ3u930/PPP17+WmJhIkiQ1ucxdS90qPvJLTEzE7bff\nDpPJBODKtb/x48ejY8eOfDrmI0899RTGjRuHuXPnYsWKFdBqtcjMzMQrr7yCf//73ygrK1M6xKDm\npptuwuXLl6HRaBAXF4fIyEikpqZi06ZNUKlUfMrrA4IgYMSIEdi1axfeeOON+tdvv/12OBwOVFVV\n+fwdildvO3HiBJKTk3H+/HlERUXh2LFjSEhIgNFoxIwZM1BaWop3331X6TCDjt27d+PWW2+FJElQ\nq9V48cUXMWLECBQVFSEmJgYbNmyAVqttdJ2VuXaio6Oh0WhQW1sLURRRUFCA9957D6WlpSgoKED/\n/v2xZ88epcMMSo4ePYobbrgB0dHR9a+9/PLLGDVqFJYuXYrp06dj/vz5vn1JIAxvRVEkm81GREQZ\nGRmUm5tLlZWV9MADD/hleBsO7YduNBoNHT9+nIiIZFmmOoqLi8lqtZLH46FZs2Y1KqzDbq/Nb10r\nLS0lSZLI5XJRcXExjRgxgqZMmUKPPPIIbd26lTIzM9lvK9wKgkCSJJHdbiedTkd6vZ6efvrp+oWO\n7XY7SZJEarXaJ7eKj/yAKwWg605764pqr1y5sr4oNNMyvF4vFixYgA8++ACyLKO6uhqXLl1CVlYW\nVq9ejR49eiAnJwfV1dVKhxrUJCUlQa1W11+eqSu3+MUXX2DTpk3Izc1VOMLghIjg9XohiiIcDkf9\nf9eh1+sBwPfLCkofPX/cVq9eTTU1NT/5OaWPVoHWrsXtyJEjyWq1UlFRUaO6EuzWd7+CIJDFYqHa\n2lrS6/Xs1we38fHxFBkZSaIo0sqVK+tLBKxZs4Y6dOjQbP9tyfcqfsPjx4wZMwbTpk1TOoyQZNCg\nQTAYDCgtLa3rdIwfMRqNiIqKwscffwyn06l0OEGN2WyGzWaDWq3GgQMH4Ha7cfPNN+M3v/kNSkpK\n/NN/lc7wP2wxMTFUWFh41VEJ+OjZKrcAaNeuXVRcXEx33nknu20Dv4WFheR0Oqlv377s109uhw4d\nSpIk0dixY0kURb/mhYAa+RER1q9fz3NN24i0tDTU1NTg1KlTSocScqhUKnTo0AFlZWXs14/88pe/\nhEqlwjfffOP3R98C4oZHHXPmzEFaWho/H9UGREREwGAwQBAEFBcXKx1OyNGhQweYzWZMmDDBpylX\nTEMmTJiA5cuXX7UaXmsJqOptJpMJBw8eRI8ePX4yyxOvidYAXm+ubWG/bQeXrmwh3IEawm7bFvbb\ndoRF8mMYhgkUAuqGB8MwzPWCkx/DMGEJJz+GYcISrt4WIrDbtoX9th1cvY1hGOY6EjDJTxAEaLVa\n3HPPPRg1ahQ+/PBDEBGqq6sxZMgQvPjii4iPj1c6zKDkpptuwm9/+1vs378f48ePx/Dhw+F2u1Fd\nXY2VK1fiySefRIcOHZQOM2jp1KkT+vfvj4ULF2LJkiX16/sxvrNz506sX78eZrMZdrsdVqvVfxsP\nlDl8Go2Ghg4dSuXl5bRixYoG69Bt2rSJXn31VZ4f2Qq3Wq2WzGYzud1uslgs5PF4yOv1ksvlIqvV\nSkePHqVvvvmmwUoZSu9LILbm/Op0Otq9ezc5HI769eZkWaa5c+eSVqvlub0+uJ0xYwZJkkSyLNPn\nn39Od911F8XExPhtbm9A7GTdQqYul4tKSkrI4/HQgQMHaMKECSSKIomi2GixA6X/wQKtNeU1Li6O\nNmzYQF6vl/Lz82nMmDEUGRnZwKVWq220KKTS+xKIrSm/giBQTU0NeTweeumll5qt2cHJr+Vu9Xo9\nHT58mLZv337NXlvqVvG5vSaTCXq9Hjk5OSgrK8Nvf/tbrnXqJx544AEAwJYtW7Bw4UJs2rSp0Wfc\nbvf1DitkiIqKQmRkJKqqqvD+++9DkiSlQwoZdDodTCYTjhw5gk6dOuHixYv+/xKlM/w777xDlZWV\n1Llz52vO7uCj51XdiqJIsbElqkeWAAAckElEQVSxZLFYyGq1Nns6zG5b57euRURE0Pbt22nx4sX1\no2lBEMhoNF51+SWl9yXQWlOO1Go1bdq0iYiIBg0aRLt27SKbzUZer9dvfVfxGx7PPfccDh8+jHHj\nxjWqIZuamqpQVMGN0WiEKIoQRRHffvstNBoNYmNjAQCiKEKr1eJ3v/udwlEGP3a7HY8//jh69eqF\ntWvXIikpCTt27MBtt92mdGhBjyzL9UuDffDBB8jMzERERARUKhXOnj2LnTt3+v4lSmd4AGS32yk7\nO5ssFgsVFhYSEdHu3btp5syZXMColW61Wi1NnTqVnnvuOdq/fz8dP36cTp06RV6vl7Zt20YpKSnU\ntWtXGjduHKlUKnbbQr91rXv37lRaWkpVVVVks9nql1tfsWIFZWVlcd9tpVtBEKi4uJi8Xi9Nnz69\nwUja4XAQXfkffXKr+E4CoDVr1tTf9JBlmTweD9ntdrLb7dyBfHD7wgsv0OOPP05nzpyhnJwcMpvN\n5PV6ac+ePbRt2zZavHgxxcTEUHJyMrtthV8A9PTTT9PYsWOpR48epNFoKCYmpr5yXlVVFffdVro1\nmUzkcrloz549ZDQaG7zXtWtXoiv/Y/Anv8jISJIkiTweD1mtVnrggQfo7Nmz5PV6ye12cwdqpVtR\nFCkxMZF+9atfkcFgIK1WS8nJyZSZmUknT56kyspK6t27N82bN4/dtsIvABo7dizpdLpGoxaiK08v\ncN9tvdsNGzZQREREo9d79OhBsiz77Dag1/PzeDwQRbHJZe2Jpwg14Mduo6KiYLfbIctykytjp6am\n4siRI1i3bh2eeuop1NbW1r/HbhvT0r5bVVUFlUqFmJiYRu+x34a01K3X64UgCFCpGt+yaIlbxW94\nNIcgCBBFES6XS+lQgg6NRoNhw4bhww8/bHbmxsCBA6HRaLB169YGiY/xD99//z2OHz+udBghSVNJ\nr1UoPbwVBKHBQ7c6nY5GjRpF3333HVmt1kYX48GnDtfk1mQyUWVlJRUVFdH+/fspISGB9Ho9RUdH\n069//WsiIlqxYkWTlfKU3pdAbE31wbr+GxsbS4IgkEajoU6dOtEdd9xBLpeLZs+ezX3XB7cAyO12\nU3l5OeXn55PZbCaHw0E2m63RdcDWuFV8Jz/88EOKi4ujjIwMiouLo1OnTlFhYSG5XK4mz/e5A127\n23vvvZc2bNhAREQej4feffddOnfuHEmSRLW1tRQZGcluffCL/13zc7lc9Mknn9D8+fPp6NGjtHfv\nXsrNzaWUlBT264NbAPXTXIuLi0mWZbJardStW7dGs5KCMvmdP3+ePv30U6quriZJkshqtVKvXr0o\nKSmpWSHcgVrWgbp160arVq2iM2fO0IgRI+iLL75gt37yKwgC9ezZk2688UayWq20e/duevDBB9mv\nH9wCIJVKRcnJyXTfffeRRqNp9oDdGrcBfcPjahBfNG4Au21b2G/bwev5MQzDXEe4ehvDMGEJj/wY\nhglLOPkxDBOWcPJjGCYsCbjqbYcOHUJiYiK6du3a5LSsOviOWUP4bmTbcq1+VSoV3nzzTYiiiOnT\npzf5GfbbEL7b+z9uvvlmlJaWwmg0Kh1KSKLT6WC1WhEXF6d0KCGHIAiYOXMmnnnmGUycOFHpcEKK\nDRs2oLi4GCtWrPDbNgMq+SUlJcHlcqF///4837QNSE9Ph9VqhdFoRE1NjdLhhBx//vOf8cQTT+Cl\nl15qckEDpnWIooj+/ftDFEV8/PHHfttuwDzkrFKpkJWVhaqqqiZrTfwYPnVoyLWcOpw6dQppaWmo\nra1FdHR0s59jt435Kb+CIODSpUsYOHAgCgoKrrot9tuQq7kVBAE33XQTTpw4gaqqKqSkpMDpdKK5\nvNUSt4oXMKpj9erVGDlyJB8x24jhw4cjOTkZ/fv3x/79+5UOJ+T49ttvkZCQgIyMDBQWFjb742Ra\nRr9+/bB+/Xo4nU4MHToULpfLf24DZQ5fQUEBlZeXX3XeHlo5hy8c2tVcpaenU05ODqWlpV21sA67\nbZ3fW265hbxeL7377rs/OfeU/bbM7dKlS6myspKGDRvW7ApPrXUbEDuZlZVF+fn59OGHHzZ4XRTF\nZmt2Kv0PFmjtah3CZrNRaWlpg9dMJlOTy1mx25b5FQSB7HZ7/crCarWa1Go1GY1GGjZsGPv1wW1m\nZiY5HA4qKCiod1u3hJher/fZbUDc8Fi/fj0qKyuh1WqhVqthMpmgVqsxZMiQOjmMD5SVleHEiRMQ\nBAEajQZdunTBmjVrIIqi0qEFPTqdDqIo4r333kNUVBQee+wxvPHGGxgwYAAOHDigdHhBS1RUFJ54\n4gkQEU6ePImuXbti6NChePLJJ7Fx40ZMnTrV9y8JhAxvMplIlmWaNWsW1dbWUk1NDXk8HqqurqbM\nzEw+evrgNi0tjaxWK+Xl5VFZWRlJkkRut5v2799PO3fuZLc++BUEgR5++GGSJImio6NpzJgxdPHi\nRXrrrbfo9ddfJ5vNxn5b6Xbt2rUkSRJVVVXR73//e8rOziabzUaSJNW35OTkRpdxWvK9ATHyMxgM\nICJUVlbi9OnTEAQB58+fh8FgwH333ad0eEFNdHQ0vF4vTp06BYfDAbPZjPPnz6N9+/b1tXyZ1qHT\n6fDHP/4RkiRhypQpcLlcmDFjBmbPno2//vWvsNvt/DxlK+nTpw9kWYbX60VhYSFUKhVcLhfsdjuA\nK/V9Hnzwwbrk2SoCIvmlpqaiuroaBoMBBw8eREZGBgYNGgSv14sxY8YoHV5Qc//992Pnzp2YPHky\nfvvb3+KTTz7BgAED4HK5YLValQ4vqImPj0e/fv2g0+nwj3/8A1u3bsXnn3+O6upqEBEcDgdGjhyp\ndJhBSXZ2NjweD3bt2oXY2Fio1WoUFBTgjTfeQE5ODsrLyzF69Ogmi5tdKwHxqMvp06eh0+nwu9/9\nDrfffjsyMjIwYsQI2Gw25OTkKB1eUJOcnIxnnnkGZWVlKC0txeXLlzFlyhSYTCYsWbJE6fCCmqKi\nIlRUVCA+Ph5HjhyBx+NBQkIC7rrrLnzwwQcwGAzYuHGj0mEGJc8//zxEUYTRaMTcuXPx2muvQa1W\nQ61Wo0uXLjAYDHj55Zd9Gvkpfm6P/93V3bNnD/3hD3+gyspKqqqqop07d1719rbS1ykCrTXn9vz5\n85STk0NRUVE0Y8YMevvttyknJ4c2b95MWq2W3frod9KkSZSXl0dms5lKSkqI6Eq93pqaGnrggQfY\nbyvdqtVqWrVqFdntdrJarWS1WslisVB+fj5VVFTQokWLKCkpqdETCy36XqV38oc7O2HCBNq/fz+d\nP3+eJkyY0OxnuQNdu9u0tLT6IjCSJNGRI0eavYnEblvuVxAEGjduHLnd7voaHu3atWO/ProVBIG0\nWi1t376diouLyWazkdVqpfz8fOrdu7dfHoELmOltLYV4ilADmnMrCAI2b96MO++8E0uXLsUf//hH\neL3eq26L3TaG+27b0ZxbrVYLQRDgdrthNBphs9l+mDCbpCVuOfmFCOy2bWG/bQcvacUwDHMd4eTH\nMExYwtXbGIYJS3jkxzBMWMLJj2GYsISTH8MwYUnAVW+7VvhxgYaw27aF/bYd/KgLwzDMdYSTH8Mw\nYQknP4ZhwhJOfiGKRqNBSkoKvv/+e3zzzTcQBKHB2mcvvPACIiIicNttt8FgMCgYafBS51Sj0SA6\nOhr/93//h1mzZuGJJ55AQkKC0uExP4WSqzdMnz6diIi8Xi/VIcsy5eTk0KpVq2jEiBG8MkYr3b76\n6qskSRK5XC76IbIsk8lkIpvNRkRXVnrZvHkzu22h35iYGJJlmTweD8myTLIsU3V1NU2cOJFmzJhB\nJ0+epFdeeYX7bivciqJI2dnZVFhYSB6PhyRJojo8Hg95vV46dOiQz24VXczUaDQiPz8f69evx/vv\nv4/Nmzdj/vz52LhxI06fPg29Xo+tW7cqGWLQ8tVXX2HixIlwu93o1q1b/euCIKC2trbB3wMHDlQi\nxKAmOTkZZWVlWLRoET788EP06dMHmzZtgizL6NixI86cOYPBgwcrHWbQIQgCEhMTYbFYYLFYsHnz\nZowcORIWiwUFBQVYvHgxpk2bhu+//973L1MywxsMhiazd0ZGBrlcrqvW6VT6aBVorSlHer2eevfu\nTXl5eXT27Flyu91ks9nq1/cjInK73fTII4+w21b4ba6ZTCbKz8+nL7/8kvtuK9wKgkB6vb7J378g\nCFRaWkpr16712a2i1/wcDkezr69duxYRERHXOaLQwul04ujRo+jWrRt+9rOfwWg0oqKiov7an8fj\nwb59+7B69WqFIw0dVCoVMjMzERMTg8jISKXDCUqICE6nE7IsN3pPp9PB6XSiurraP18UaEdPnU5H\nVVVVlJiYyCM/P7qtqqqiOgoLC2n69Ons1o9+AdCxY8eIiMjhcNDkyZPZrx/d9uzZkxwOBy1btox0\nOl1wj/yaY8+ePYiNjYXT6VQ6lJAiJiamrrPBbDZjwYIFCkcUetSdrdTW1mLx4sUKRxM6CIKAN954\nA3q9Hp999hlcLpfP2wyI5CeKYoO/L126BADo0qWLEuGEJO3atWvwuMvHH3/8k8vZMy2nXbt28Hg8\nWLVqldKhhBSCIECSJHi9Xhw4cMA/Gw3E4W1NTQ3Rlf+BT3v94PZXv/oVWSwWIrryWFFFRQW79aPf\nulZWVkYul4v+9Kc/Naoqxn59c7thwwYqKSmhOXPm+K3vBsTI74eIogitVqt0GCGFRqNBVFQUAODf\n//43hg8frnBEoUm7du1QWVkJtVpd96Nm/MRdd92FsrIyVFZW+m+jgZbhExISyOl00sWLF3l04ge3\nnTp1ojq8Xm+zJf/Ybev8Alceyn3mmWfI6XTSyJEjrzrqY78tc6vVamnXrl1ksVjopZdeavbxuNa4\nDbiRX92FzPj4eIUjCX46derU4GHQ8vJyREZGQqUKuH/2oCYyMhKDBg3C6tWrsXPnzrofNOMHUlJS\noNPpsGbNGqxYscK/N0EDJcPXtfT0dJIkiU6dOsVHTx/dlpaWksPhIKIr09iu9ugQu22537p25MgR\ncjqdPzniY78td7tu3TpyuVw0atQov7sNuLq9Go0GNpsNsiwjOjq62VvaxAtCNqApt1u2bMGAAQPw\n1ltvYcuWLTh//jzMZvNPbovdNuZqfdfr9cJut8NkMl3TtthvQ5pzK4oizGYzNBoNTCZTkw89/5iW\nuA245HetcAdqCLttW9hv26GUWy5dyTBMWMJXvhmGCUs4+TEME5Zw9bYQgd22Ley37eDqbQzDMNcR\nTn4Mw4QlAZP81Go1CgoK4HA4eBHTNkIQBBgMBkyePBmyLOPChQvQ6/VKhxUSCIKA8vJy2O12FBcX\nw+FwIDs7G1qttkHhKKblaLVaeDwefP/997j77rv9Nzc9EJ7kHj58eIMiJQ6Hg7p27cpPyfvpKfm6\nlpWVVe9ZlmWy2Wz01VdfsVs/+F28eDFJkkRer5ccDgdVV1dTYWEhdenShZKTk9lvK90ajUaaPHky\neTweOnfuHNXW1lJVVRU9++yzPucFxXeypKSErFYrOZ1OGjp0KE2fPp1KSkrI4/GQ3W6n3r17c/Lz\nsQMNHTq0vhKW0+mkQ4cO0bJly8jr9dKqVasa1UpQel8CsV3N70cffURms5l2795NHTt2JFEUKTU1\nlfbs2UOrVq2iOXPm0IQJE9hvC9xGR0eT2Wwmr9dL5eXllJKSQjt27CC73V5fg8bXvKD4aW9CQgKM\nRiOeffZZ7Nq1C++88w4yMzORk5MDg8GAnTt3Kh1i0JOVlYWYmBgcO3YMS5YsARGhU6dOEAQBTzzx\nxDVNG2Kap67uscvlQm1tLQwGA0RRhEqlwpAhQ1BSUoLs7GyFowwuvv32W8TFxQEApk+fjoKCAhQW\nFkKv19clTN9ROsNbrVay2WykVqsJAPXo0YOcTicREc2bN49Pe31wC1yphJednU1er5cefvhhmjlz\nJu3evZtkWaZ9+/axWx/9AqCIiAjKzMykXbt20bBhw2jAgAFUWFhILpeLUlNTeWTdCrdDhw4lWZbp\nwIED1KFDB1Kr1fTBBx+Qy+Uil8tFJpMp+E973W43FRYW0t/+9je64447yOFwkMfjIZvNdtVrLEr/\ngwVaa85TRUUFud1ustvtdOzYMbJYLOR0Oslut1NKSgq79dEvAOrVq1d9ScUvv/yS8vLyyGw2N9uH\nld6XQGtNORJFkWRZpuLiYsrKyqK4uDiqrKwkWZbpqaeeIlEUgz/5AVdueIwfP56IrlQVKy4upsLC\nQk5+Prq98cYb6Yd4PB6SJImsVit99913dPfdd7NbH/zWtdjYWNq5c2e9Z0mSaOfOnX4ZnYRDa86r\nWq2mF154gdxuN3m9XiIiMpvNzR60W+pW8Wt+ALB9+3ZcvnwZq1atQl5eHoYMGQKNRoOysjJ07txZ\n6fCClvPnz+Py5csAAFmW8eWXX+LVV19FdnY2Dhw4gG3btikcYWiwceNGZGRk1P9tt9vx8ssvw2az\nKRhV8OPxeLB161YUFRVBpVLB4/EgMjISQ4YM8cv2r+v0tquxefNmXLx4EdnZ2ejYsSPy8vLQp08f\nxMfH49KlS3VHCKaFREVFQa1WQ5IkiKKIPn364LHHHsPEiRMhSZLS4QU1oiji1ltvRWZmJlQqFdxu\nN0RRhM1mw/79+/lGkh/o06cPoqOjUVtbi5qaGnTq1Kn+BpPPBMLwFgDNmjWLiKjBxWFZlunixYs0\nYcIEmj17Np86tNBtZGQkDR8+vP5vlUpFvXv3piNHjlDPnj39Uvg5XNqPHSUkJJAsy+T1eql9+/aU\nkJBA+/fvJ7fbTVarlWJjY/mSTSvd/rBVVVVRfn4+JScn08KFC+nrr78mq9UaOtf86hIdXflQfSsv\nL6e77rqLvvjiC+5ArXBbU1ND06dPJ41GQ4IgUHx8PFmtVsrNzaXVq1dz8vPB75o1a4iIqLy8nARB\nII1GQ5IkkSzLVFlZyaUrfXBb1/r27Us7d+6kpKQkAq48T3ns2DE6duwYpaamhsY1PwCw2Ww4ffo0\nXnvtNUyYMAE5OTnwer0YOHAg7rvvPqXDC0qioqIwb948SJIEvV6PM2fOwOl0IiUlBd9++y1SU1OV\nDjEo0el0sFgssFqtcLlcGD9+PKqqqqBSqZCfn4/777+/7kfNtJIOHTpg//79GD9+PIxGI5YvX44R\nI0YgNTUVO3fuxLlz53z/kkDI8ABo/fr1VIfFYiGr1UqSJNGDDz7IR89WuvV4PGS1WumRRx6hTz75\nhDweD3k8Htq6dSvFxcU1OzpRel8Csf3Qj16vp2HDhpHL5aJTp07Vj/i+/vprysjIaPRcH/u9drc/\nbA6Hg9xud/3jWXXTMf01qg6InaxrRqORvvvuO7rrrrto+PDhfOrgo9sBAwaQ1WolWZaptraWtm3b\nRvfddx//OP3kt7VN6X0JtNacp9GjR5MkSVRTU0Pz588nk8nk15rIXMAoRGC3bQv7bTt4MVOGYZjr\nCFdvYxgmLOGRH8MwYQknP4ZhwhKu3hYisNu2hf22HXzDg2EY5jrCyY9hmLCEkx/DMGFJwCW/zMxM\neDweyLIMWZZRWVmpdEghQ0JCArp164bMzEwup+hH3nnnHZSUlGDLli2w2+04ffo0Zs6cidGjRysd\nWsgwevRoHDhwAPn5+SguLsaoUaN832ggTGMBQGPHjqWqqiqyWq30yiuv0NSpU6mOpj6v9JScQGtX\ncztp0qT6+dJ1q+eo1Wqe2+sHvwMGDCC32021tbW0fft2On36NDmdTvJ4POR2u9mvD24B0J49e6im\npobsdjvl5ubW92GXyxUac3sFQaDq6mpyu90N5p1WVFT4ZSfDoTXlSKVSUefOnWnv3r3kcrnIZrNR\nHTxv2ne/JpOJjh8/Tna7naZOnUrx8fHUoUMH+te//kUnT56kCxcusN9WugVAOp2OLBZLfR1vrVZL\ncXFxVFpa2myZi5Z8r+KnvSkpKaisrITJZMLNN99cv/qtIAiQZdlvS1aHI48++ijeeustSJKELVu2\nAAAqKipw6dIlqNUBs4h30JKTk4Pu3btj5MiRWLhwIcxmM0pKSjBjxgy8/vrr2LZtW4Pl7ZlrZ/To\n0di7dy+Ki4vRsWNH5OfnQ5ZljB07Fi6XC8uWLfP9S5TO8AMGDCCXy0XV1dU0aNAgUqlU1KVLF3r1\n1VdJkiRasGABGQwGPnq2wu3p06epsLCQFi5cSHfccQdZrVayWq20YcOGJp2y22v3m5ycTA6HgyRJ\norS0NNJqtSSKIhkMBho+fDjl5OSQ0+nkvtsKtwAoOzubysvL6ZFHHiGtVkvdunWjdevWkdvtpk2b\nNlFGRobPfVfxVV0EQUDnzp2Rl5cHSZKwfft2eL1e3H333fjLX/6CmpqaJrM88YOiDWjK7eLFi/GL\nX/wCKSkpqK6uhtFohFqthsFgQHJyMoqKiprcFrttzI/9jhs3DnPnzsXXX3+N8ePHQ5IklJeXIyEh\nAUajEYIgoKqqComJifjxb4z9NqSpvltSUgJJklBVVQWLxYLBgwcDuFI3pX///jh27BicTmejbbXE\nreLnPkSEgoIC3HTTTSgrK8Py5cvRr18/iKKIFStWwG63Kx1i0JKSkgKPx4M1a9ZAFEXs378fr732\nGgDAYDBAEIRGP0zm2li3bh3Gjh2LmJgY6PV66PV6uN1uREVFQRAESJKEAwcOsN9WYjQaodPp0Llz\nZxARSkpKkJiYCADo0aMHcnJyfP4OxZNfHXl5eQCAbdu24Ze//CWsViscDgccDofCkQUvf/nLXzB8\n+HDMmDEDWVlZcLlcqK6uhsFggE6n4x+mD0iShCeffBI2mw3nzp1DUVERRowYgXvvvRderxcff/wx\nunXrpnSYQUtFRQUAoF27djh48CA+//xzvPXWWwCAs2fP+qcsqNLn9j9uNpuNioqKfvJzSl+nCLT2\nU760Wi2lpKTQmTNnyGq1NltQm9223K8gCPTggw+Sx+Mhr9dLDoeD4uPj2a8f3A4ZMoQeeughKiws\nJK/XSwUFBX7LCwEz8gOA4cOHw2AwoGfPnkqHEnK43W58+eWXuOGGGyAIAhfU9iMPPfQQZsyYAVEU\n4fF4UFZWBovFonRYIcGePXvQrl07dOjQAYIgQKfT+W/jgZLhP//8cyIievHFF39y1Ac+erbIbV3r\n3LkzXbhwgc6fP09arZbd+sGvIAhktVrJ4/HQN998Q+3ateMaKX5yW9dSUlKosrKSzp8/T2PHjvWb\nW8Wf86vjnnvuARFhz5490Gq1SocTkthsNrhcLnTs2BGSJCkdTkhgMBhgMBggiiI+/fRT2O12njro\nZ+655x5ERUWhtLQUubm5/ttwIGT4jIwM8nq9ZLFYrmnUBz56tvjoCVyZQmi326mqquqqoxOl9yUQ\nW3OunnnmGfJ6vc3ORGK/rXcLXLlWbbfbaffu3WQ0GkkUxdC65te/f3988cUX/j2fZxpRVFSEgwcP\nQqvV1nU6xkcOHDiAsrIy/PWvf1U6lJBErVbDYrHg8ccfh91ur58B5g8Uf8i5tRA/KNoAdtu2sN+2\nQym3XL2NYZiwJGBueDAMw1xPOPkxDBOWcPJjGCYs4eTHMExYwsmPYZiwhJMfwzBhCSc/hmHCEk5+\nDMOEJZz8GIYJSzj5MQwTlnDyYxgmLOHkxzBMWMLJj2GYsISTH8MwYQknP4ZhwhJOfgzDhCWc/BiG\nCUs4+TEME5Zw8mMYJizh5McwTFjCyY9hmLCEkx/DMGEJJz+GYcKS/wdjOpx6pXFKxAAAAABJRU5E\nrkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f41ed77fe90>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "num_image = 4\n",
    "for digit in range(10):\n",
    "    for i in range(num_image):\n",
    "        latent_z = mx.nd.random_normal(0, 1, shape=(1, latent_z_size), ctx=ctx)\n",
    "        label = nd.one_hot(nd.array([[digit]]), 10).as_in_context(ctx)\n",
    "        img = netG(nd.concat(latent_z, label.reshape((1, 10))))\n",
    "        plt.subplot(10, 4, digit * 4 + i + 1)\n",
    "        visualize(img[0])\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For whinges or inquiries, [open an issue on  GitHub.](https://github.com/zackchase/mxnet-the-straight-dope)"
   ]
  }
 ],
 "metadata": {
  "anaconda-cloud": {},
  "kernelspec": {
   "display_name": "Python 2",
   "language": "python",
   "name": "python2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.12"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
